The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.
Various techniques exist for an information retrieval computer system to present documents to a user that match a given query. These techniques generally operate by computing ranking scores for the matching documents according to a relevance scoring model. The matching documents are then presented to the querying user in order of their ranking scores. For example, the matching documents may be presented to the querying user as a search result in a web page or other graphical user interface.
A search result may provide digests, synopses or other summaries of the matching documents. The summaries may be presented in the user interface in order of the ranking scores of the corresponding documents. For example, the summary of the matching document with the highest or best ranking score for the most relevant matching document may be presented near the top of the user interface above the summaries for relatively less relevant matching document.
A summary presented to a user as part of a search result may also be actionable by the user with user input. For example, by clicking on or otherwise directing user input toward a summary for a matching document, the user may be able to command some action on the matching document such as downloading the matching document, viewing the matching document, opening the matching document, or some other action appropriate for the matching document.
For text-based information retrieval systems, the ranking scores computed for matching documents are often a function of the query terms in the query. For example, the ranking score for a matching document may be a function of the variance between the matching document's term frequency (TF) and the matching document's inverse document frequency (IDF). The matching document's term frequency (TF) may be a function of how many times each query term appears in the matching document. The matching document's inverse document frequency (IDF) may be a function of how many items in a corpus of documents each query term appears. The variance may be computed as the product of the matching document's term frequency (TF) and the matching document's inverse document frequency (IDF).
A term frequency-inverse document frequency (TF-IDF)-based ranking function may normalize TF and IDF according to a probabilistic information retrieval framework to compute ranking scores that more accurately reflect relevance. For example, the widely-used BM25 family of ranking functions dampen the effect of a relatively large term frequency (TF) and adjust the term frequency (TF) based on the length of the matching document relative to the average length of documents in the corpus to which the matching document belongs.
In addition to a query-dependent score, such as a BM25 score mentioned above, a ranking function may incorporate a non-query dependent (query independent) score. A query independent score may reflect the relevance of a matching document based on various metrics that are independent of how well the document matches the query. One example of an algorithm for computing a query independent score is the PageRank algorithm which computes a query independent score based on the number and the quality of documents that link (refer) to a matching document. There are many other types of query independent scores and different algorithms for computing them.
One challenge facing designers and operators of text-based information retrieval systems is what weights to apply to each of the individual query-dependent and query-independent field scores on which the ranking function is based. As a simple example of the challenge faced, a query-dependent score may be based on two individual query-dependent field scores. One of the individual query-dependent field scores may be, for example, a BM25 score on the title of a matching document while the other individual query-dependent field score may be a BM25 score on the body of a matching document. Intuitively, the designer or operator may configure the ranking function to assign more weight to the BM25 score for the title relative to the weight assigned to the BM25 score for the body on the assumption that users will consider a query term match on the title to be more relevant than a match on the body.
To assign the weights, the designer or operator may engage in trial and error. For example, the designer or operator may submit what are believed to be representative queries to the information retrieval system and “eyeball” the search results to see if the most relevant items are presented as expected. If the search results are not as expected, then the weights may be adjusted and representative queries submitted again to verify the adjustment. This process of trial and error may be repeated until the assigned weights “look” optimal. Unfortunately, this process can be tedious, time consuming, and error prone, especially for a relatively large corpus of searchable items (e.g., millions of items or more) that undergo frequent changes or edits, or where there the ranking function has more than a few weighted components.
Thus, there is a need for techniques for more efficiently determining optimal weights to use in a ranking function. The present disclosure provides techniques that address this and other needs.
While the present technology is amenable to various modifications and alternative forms, specific embodiments are shown by way of example in the drawings and are described in detail. It should be understood, however, that the drawings and detailed description are not intended to limit the present technology to the particular form disclosed. The intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present technology as defined by the appended claims.